Robotic systems and methods for identifying and processing a variety of objects

ABSTRACT

A robotic system is disclosed that include an articulated arm and a first perception system for inspecting an object, as well as a plurality of additional perception systems, each of which is arranged to be directed toward a common area in which an object may be positioned by the robotic arm such that a plurality of views within the common area may be obtained by the plurality of additional perception systems.

PRIORITY

The present application is a continuation of U.S. patent applicationSer. No. 16/800,587, filed Feb. 25, 2020, which is a continuation ofU.S. patent application Ser. No. 15/982,238, filed May 17, 2018, nowU.S. Pat. No. 10,621,402, issued Apr. 14, 2020, which is a continuationof U.S. patent application Ser. No. 15/260,837, filed Sep. 9, 2016, nowU.S. Pat. No. 10,007,827, issued Jun. 26, 2018, which claims priority toU.S. Provisional Patent Application Ser. No. 62/217,200, filed Sep. 11,2015, as well as U.S. Provisional Patent Application Ser. No.62/269,640, filed Dec. 18, 2015, the disclosures of which are herebyincorporated by reference in their entireties.

BACKGROUND

The invention generally relates to robotic and other sortation systems,and relates in particular to robotic and other sortation systems thatare intended to be used in dynamic environments requiring the system toaccommodate processing a variety of objects.

Many order fulfillment operations achieve high efficiency by employing aprocess called wave picking. In wave picking, orders are picked fromwarehouse shelves and placed into bins containing multiple orders thatare sorted downstream. At the sorting stage individual articles areidentified, and multi-article orders are consolidated into a single binor shelf location so that they may be packed and then shipped tocustomers. The process of sorting these articles has been done by hand.A human sorter picks an article from an incoming bin, finds the barcodeon the object, scans the barcode with a handheld barcode scanner,determines from the scanned barcode the appropriate bin or shelflocation for the article, and then places the article in theso-determined bin or shelf location where all articles for that orderare placed.

Manually operated barcode scanners are generally either fixed orhandheld systems. With fixed systems, such as those used atpoint-of-sale systems, the operator holds the article and places it infront of the scanner so that the barcode faces the scanning device'ssensors, and the scanner, which scans continuously, decodes any barcodesthat it can detect. If the article is not immediately detected, theperson holding the article typically needs to vary the position orrotation of the article in front of the fixed scanner, so as to make thebarcode more visible to the scanner. For handheld systems, the personoperating the scanner looks for the barcode on the article, and thenholds the scanner so that the article's barcode is visible to thescanner, and then presses a button on the handheld scanner to initiate ascan of the barcode.

Other ways of identifying items by barcode scanning require that thebarcode location be controlled or constrained so that a fixed orrobot-held barcode scanner can reliably see the barcode. Automaticbarcode scanners also involve either fixed or hand-held systems, and thesame principles apply. In the case of barcode scanners typically used inindustrial applications, the possible positions of barcodes must betightly controlled so that the barcodes are visible to the one or morescanners. For example, one or more barcode scanners may be placed infixed locations relative to a conveyor or series of moving trays so thatthe scanners may scan objects, typically boxes, as they pass by thescanners. In these installations the range of placement of the barcodesis comparatively limited as they must be on labels affixed to one offour sides or top of a box, which also needs to be presented atorientations suitable for scanning. The detected barcode is thenassociated with the immediate section of the conveyor or is associatedwith the particular moving tray in which the object had been placedprior to scanning.

In all of these cases, the systems employ sensors, cameras or laserreflectivity sensors, as well as software to detect barcodes and decodethem. These methods have inherent limitations that include the range ofdistances of orientations relative to the detection system, over whichthey are able to reliably scan barcodes. Firstly, the barcode must befacing the scanner; secondly the range to the barcode must be such thatindividual elements can be reliably distinguished; and, thirdly, thetilt and skew of the barcode must be such that individual elements canbe reliably distinguished. The types of sensors employed, and therobustness of the software detection and decoding schemes determinethese performance parameters. There remains a need, therefore, for anobject identification system for a robotic system that is able toaccommodate the automated identification and processing of a variety ofobjects in a variety of orientations.

SUMMARY

In accordance with an embodiment, the invention provides a roboticsystem that include an articulated arm and a first perception system forinspecting an object, as well as a plurality of additional perceptionsystems, each of which is arranged to be directed toward a common areain which an object may be positioned by the robotic arm such that aplurality of views within the common area may be obtained by theplurality of additional perception systems.

In accordance with another embodiment, the invention provides a methodof identifying an object in a robotic system including an articulatedarm. The method includes the steps of inspecting the object using afirst perception system and providing an inspection response signalrepresentative of whether the item has been identified, moving theobject to a plurality of additional perception systems responsive to theinspection response signal, and inspecting the object using theplurality of additional perception systems, each of which is arranged tobe directed toward the object in a common area from a different view ofa plurality of views of the object within the common area that may beobtained by the plurality of additional perception systems.

In accordance with a further embodiment, the invention provides aperception system for assisting in identifying an object. The perceptionsystem includes a plurality of perception units that are each positionedto be directed toward a plurality of locations along an object path thatan object may take as the object travels through the perception system.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description may be further understood with reference tothe accompanying drawings in which:

FIG. 1 shows an illustrative diagrammatic view of a system in accordancewith an embodiment of the present invention;

FIG. 2 shows an illustrative diagrammatic view of a portion of thesystem of FIG. 1 that includes a plurality of perception units;

FIG. 3 shows an illustrative diagrammatic view of perception image dataof a set of objects to be sorted as presented at an infeed station in asystem in accordance with an embodiment of the present invention;

FIG. 4 shows an illustrative diagrammatic view of a single simulatedhold of an object to be identified in accordance with an embodiment ofthe present invention;

FIG. 5 shows an illustrative diagrammatic view of multiple overlappingsimulated holds of the object of FIG. 4 ;

FIG. 6 shows an illustrative diagrammatic view of simulated overlappingpositions of many barcodes;

FIG. 7 shows an illustrative diagrammatic view of a configuration ofperception units based on the modelling of FIGS. 4-6 ;

FIG. 8 shows an illustrative diagrammatic view of a system in accordancewith a further of the present invention;

FIG. 9 shows an illustrative diagrammatic isometric view of a dropperception unit for use in accordance with the system of FIG. 8 ; and

FIG. 10 shows an illustrative diagrammatic top isometric view of thedrop perception unit of FIG. 9 .

The drawings are shown for illustrative purposes only.

DETAILED DESCRIPTION

In accordance with an embodiment, the invention provides an objectperception system for the purposes of automatically sorting individualobjects in a set. In applications such as order fulfillment, objects arecollected into heterogeneous sets and need to be sorted. Individualobjects need to be identified and then routed to object-specificlocations. The described system reliably automates the identification ofsuch objects by employing both automated barcode scanners and a roboticarm. In accordance with certain embodiments, for example, the systemuses the robotic arm to hold the item in front of one or more barcodescanners so that the object may be scanned. In accordance with variousembodiments, the perception units (e.g., cameras or scanners) may lookfor a variety of codes such as indicia (e.g., barcodes, radio frequencytags, Stock Keeping Unit (SKU), Universal Product Code (UPC), lowwavelength IR (LWIR), as well as invisible barcodes and digitalwatermarks such as Digimarc DWCode, etc.).

Whereas fixed industrial scanners require that the object's barcode besituated so that its barcode is visible to a scanner, the robotic arm ofthe present invention may pick an object out of a heterogeneouscollection of objects where the barcode is not visible and scan theitem. The result is an automated barcode scanning system for arbitraryitems in a heterogeneous stream of objects that may be used toaccurately and reliably identify items.

Sorting for order fulfillment is one application for automaticallyidentifying objects from a heterogeneous object stream. Barcode scannershave a wide variety of uses including identifying the stock keeping unitof an article, or tracking parcels. The described system may have manyuses in the automatic identification and sortation of objects.

Operating in conjunction with a robotic pick and place system, such asystem of an embodiment of the present invention automates part of thesorting process, in particular the step of identifying picked objects.Instead of a person picking the object from a bin, a robotic arm maypick an object from a bin, places the object in front of a barcodescanner, and then, having obtained identification codes for the object,places the object in the appropriate bin or shelf location. Sincebarcode scanners employ cameras or lasers to scan 1D or 2D symbologiesprinted on labels affixed to articles, the barcodes must be visible tothe scanner's sensors for successful scanning in order to automaticallyidentify items in a heterogeneous stream of arbitrary articles, as in ajumbled set of articles found in a bin.

In accordance with various embodiments, therefore, the inventionprovides a method for determining the identity of an object from acollection of objects, as well as a method for scanning the barcode ofan object employing one or more barcode scanners and a robotic arm withend-effector holding the object. The invention further provides a methodfor determining the placement of fixed barcode scanners so as tomaximize the probability of successfully scanning an object held by arobot end-effector in accordance with certain embodiments, as well as amethod for determining a sequence of placements of a robot end-effectorso as to minimize the time it takes a configuration of one or morebarcode scanners to successfully scan an object, and a method forscanning the barcode of an object by employing a barcode scanner as anend-effector on a robotic arm.

An important aspect is the ability to identify objects via barcode orother visual markings of objects by employing a robot arm to pick upindividual objects and place or drop them in front of one or morescanners. Automated scanning systems would be unable to see barcodes onobjects that are presented in a way that their barcodes are not exposedor visible. Since the system uses a robot to hold the object, it caneither maneuver the object so as to make the barcode visible, or employmultiple scanners that view the article from multiple viewpoints toacquire the barcode, irrespective of how the object is held.

An automated article identification system in accordance with anembodiment of the present invention is shown in FIG. 1 . FIG. 1 shows arobotic system 10 that includes an articulated arm 12 that includes anend effector 14 and articulated sections 16, 18 and 20. The articulatedarm 12 selects items from a conveyor 22, that are either in a bin on theconveyor 22 or are on the conveyor itself. A stand 24 includes anattached first detection unit 26 that is directed toward the conveyorfrom above the conveyor. The perception system may be a perception unit26, for example, a camera, or a scanner such as a laser reflectivityscanner or other type of bar-code reader, or a radio frequency IDscanner. A plurality of additional perception units are provided on aperception system 28 (as will be discussed in more detail below withreference to FIG. 2 ).

The robotic system 10 may further include the robotic environment, atarget station 30 that includes a number of bins 32 into which objectsmay be placed after identification. A central computing and controlsystem 34 may communicate with the perception unit 26 and the perceptionsystem 28 as well as the articulated arm 12 via wireless communication,or, in certain embodiments, the central computing and control system 34may be provided within the base section 20 of the articulated arm.

FIG. 2 shows the perception system 28 that includes a plurality ofperception units 40, 42, 44, 46 and 48, as well as a plurality ofillumination sources 50, 52, 54 and 56 for use in certain embodiments ofthe present invention. Each of the perception units 40, 42, 44, 46 and48 may be, for example, a camera (e.g., 2D or 3D), or a scanner such asa laser reflectivity scanner or other type of barcode reader (e.g., 1Dor 2D barcode scanners), or a radio frequency ID scanner together withthe associated software to process the perceived data.

Generally, the system provides in a specific embodiment, an automatedarticle identification system that includes a robotic pick and placesystem that is able to pick articles up, move them in space, and placethem. The system also includes the set of objects themselves to beidentified; the manner in which inbound objects are organized, commonlyin a heterogeneous pile in a bin or in a line on a conveyor; the mannerin which outbound objects are organized, commonly in an array ofoutbound bins, or shelf cubbies; the manner in which objects are labeledwith barcodes or radio-frequency identification tags; a fixed primaryscanner operating above the incoming stream of objects; a barcodescanning station where one or more barcode scanners and illuminators areactivated when the object is held at the station; and a centralcomputing and control system determines the appropriate location forplacing the object, which is dependent on the object's decoded barcode.

As noted, the robotic pick and place system is typically a robotic armequipped with sensors and computing, that when combined is assumedherein to exhibit the following capabilities: (a) it is able to pickobjects up from a specified class of objects, and separate them from astream of heterogeneous objects, whether they are jumbled in a bin, orare singulated on a motorized or gravity conveyor system, (b) it is ableto move the object to arbitrary places within its workspace, (c) it isable to place objects in an outgoing bin or shelf location in itsworkspace; and, (d) it is able to generate a map of objects that it isable to pick, represented as a candidate set of grasp points in theworkcell, and as a list of polytopes enclosing the object in space.

The allowable objects are determined by the capabilities of the roboticpick and place system. Their size, weight and geometry are assumed to besuch that the robotic pick and place system is able to pick, move andplace them. These may be any kind of ordered goods, packages, parcels,or other articles that benefit from automated sorting. In certainembodiments, each object is associated with a stock keeping unit (SKU),which identifies the item.

The manner in which inbound objects arrive may, for example, to be inone of two configurations: (a) inbound objects arrive piled in bins ofheterogeneous objects as shown in FIG. 3 ; or (b) inbound articlesarrive by a moving conveyor. As shown in FIG. 3 , the collection ofobjects includes some that have exposed bar codes as shown at 60, 62,64, 66, 68, 70 and 72, and other objects that do not have exposed barcodes. The robotic pick and place system is assumed to be able to pickitems from the bin or conveyor. The stream of inbound objects is thesequence of objects as they are unloaded either from the bin or theconveyor.

The manner in which outbound objects are organized is such that articlesare placed in a bin, shelf location or cubby or other destinationlocation at which all objects corresponding to a given order areconsolidated. These outbound destinations may be arranged in verticalarrays, horizontal arrays, grids, or some other regular or irregularmanner, but which arrangement is known to the system. The robotic pickand place system is assumed to be able to place objects into all of theoutbound destinations, and the correct outbound destination isdetermined from the SKU of the object.

It is assumed that the objects are marked in one or more places on theirexterior with a visually distinctive mark such as a barcode orradio-frequency identification (RFID) tag or other unique identifier sothat they may be identified by a scanner. The type of marking depends onthe type of scanning system used, but may include 1D or 2D barcodesymbologies. Multiple symbologies or labeling approaches may beemployed. The types of scanners employed are assumed to be compatiblewith the marking approach. The marking, either by barcode, RFID tag, orother means, encodes a symbol string, which is typically a string ofletters and numbers. The symbol string uniquely associates the objectwith a SKU.

The primary perception unit 26 is a device mounted above the area wherethe inbound objects arrive, which scans all inbound objects foridentifying features. When inbound objects arrive in bins, the primaryperception unit is mounted above the bin. The primary perception unitconsists of one or more devices that are able to recognize visuallydistinctive marks, which may include barcodes or other identifyingfeatures, or RFID tags on the objects. Unit components may includecameras, RFID scanners, illuminators, and decoding microprocessors. Theprimary perception unit makes the first pass at recognizing items in thebin. The primary perception unit localizes any codes that it has found,and the robotic pick and place system is assumed to be able to use thatlocation to pick the item with the given code.

The perception system 28 (e.g., the secondary scanners) is an assemblythat scans any objects that the primary perception unit is unable toscan among the inbound objects, or that scans the objects to confirm theresults of the primary perception unit. In further embodiments, thesecondary scanners may be used to detect any additional indicia, whichmay either confirm the identification or may detect that more than oneobject has been grasped, in which case, both are returned to the inputinbound area. Inbound objects in a bin or on a conveyor may have labelsor tags partially or completely occluded by other objects, or the labelsor tags may not be facing the scanner. The secondary perception systemis mounted in the workspace of the robotic pick and place system so thatunidentified articles may be presented to the secondary scanner. Likethe primary perception unit, the secondary perception system consists ofone or more perception devices, which may include cameras, RFIDscanners, illuminators, and decoding microprocessors.

The operations of the systems described above are coordinated by thecentral control system 34. This system determines from perception data(e.g., symbol strings) the SKU associated with an object, as well as theoutbound destination for the object. The central control system iscomprised of one or more workstations or central processing units(CPUs). The correspondence between SKUs and outbound destinations ismaintained by the central control system in a database called amanifest. The central control system maintains the manifest bycommunicating with a warehouse management system (WMS).

During operation, the broad flow of work may be generally as follows.First, the system is equipped with a manifest that provides the outbounddestination for each inbound object. Next, the system waits for inboundobjects to arrive either in a bin or on a conveyor. When the roboticpick and place system recognizes that one or more inbound objects arepresent, the central control system instructs the primary perceptionunit to scan the inbound objects. The primary perception unit creates alist of detected markings, which includes their symbol strings, as wellas their positions in the workspace. The primary perception unittransmits this list to the central control system.

The central control system receives from the primary perception unit theperceived information including the detected markings, and from therobotic pick and place system it receives a list of unidentified butpick-able objects. The position coordinates in both lists are registeredto one another by employing calibration information about the primaryscanner and the robotic pick and place system. The central controlsystem employs the map generated by the robotic pick and place system todetermine by simple geometric means the objects that enclose eachdetected marking. Thus, the central control system associates for eachmarking the object to which it corresponds. This step is a marking-baseddata association. The central control system ranks each of the candidatearticles to pick based on a heuristic, such as choosing the top-mostobject in a pile, and so generates a candidate pick list.

Again, FIG. 3 shows an example of a marking-based data associationbetween objects identified by the robotic pick and place system andmarkings detected by a perception unit. In this instance barcode symbols(bounding quadrilaterals) are associated with the unidentified objects(shaded segments) in which the barcode symbols lie.

If there is at least one candidate pick in the candidate pick list thatis associated with a marking, the system picks the highest-ranking pick.The expectation is that the picked object will correspond to the markingpreviously associated by the central control system, and detected by theprimary perception unit. Given that this association may potentially beerroneous, the central control system runs a check. After the object hasbeen removed and separated from the set of inbound objects, the systeminstructs the primary perception unit to scan the inbound articlesagain. If the correct object was removed, then the marking associatedwith it should no longer be in the list of markings detected by theprimary perception unit. If the marking associated with the picked itemis still there however, then it must be that it picked the wrong item.If it picked the wrong item, then it puts the item back and repeats theprocess of generating pick candidates from the robotic pick and placesystem and primary scanner.

If there are no candidate picks associated with a marking, then it picksthe object associated with the highest-ranking pick. Since there is nomarking associated with the object, it is an unidentified object. Afterthe robotic pick and place system picks the item out of the set ofinbound objects, the central control system instructs the robotic pickand place system to move the object to the secondary perception systemfor scanning. The central control system instructs the secondaryperception system to scan the unidentified object.

If the secondary perception system successfully recognizes a marking onthe object, then the object is then identified and the central controlsystem commands the robotic pick and place system to transfer the itemto the outbound destination determined from the SKU, itself determinedfrom the detected marking.

If the secondary perception system is unable to recognize a marking onthe object, then depending on the configuration of the scanner, thecentral control system may command the robotic pick and place system torotate the object to try to make markings visible to the scanners, andto scan the object again. This may occur a number of times to guaranteethat if the marking were on the object, then it would be detected. Thesequence of locations and orientations of the objects are chosen so asto minimize the average or maximum amount of time that secondaryscanning takes. If the object cannot be identified or if the secondaryperception system detects non-matching product identifiers (possiblyindicating a double pick), the object may be transferred to a specialoutbound destination for unidentified objects, or it may be returned tothe inbound stream.

This entire procedure operates in a loop until all of the objects in theinbound set are depleted. The objects in the inbound stream areautomatically identified, sorted, and routed to outbound destinations.

In accordance with an embodiment therefore, the invention provides asystem for sorting objects that arrive by inbound bins and that need tobe placed into a shelf of outbound bins, where sorting is to be based ona barcode symbol. In this embodiment, the primary and secondaryperception systems are able to detect and decode barcode symbologies.

Key specializations in this embodiment are the specific design of theprimary and secondary perception systems so as to maximize theprobability of a successful scan, while simultaneously minimizing theaverage scan time. The probability of a successful scan and the averagescan time make up key performance characteristics. These key performancecharacteristics are determined by the configuration and properties ofthe primary and secondary perception systems, as well as the object setand how they are marked.

The two key performance characteristics may be optimized for a givenitem set and method of barcode labeling. Parameters of the optimizationfor a barcode system include how many barcode scanners to include whereand in what orientation to place them, and what sensor resolutions andfields of view for the scanners to use. Optimization can, in certainembodiments, be done by simulation with models of the object.

Optimization through simulation employs a barcode scanner performancemodel. A barcode scanner performance model is the range of positions,orientations and barcode element size that a barcode symbol can bedetected and decoded by the barcode scanner, where the barcode elementsize is the size of the smallest feature on the barcode. These aretypically rated at a minimum and maximum range, a maximum skew angle, amaximum pitch angle, and a minimum and maximum tilt angle.

If a barcode scanner and symbol are held upright, and the barcode symbolis facing the scanner such that the symbol is parallel to thesensor-side of the scanner, then the barcode symbol is in what is calledthe fronto-parallel plane. The angle between the fronto-parallel planeand a plane that rotates about the vertical axis is the skew angle. Theangle between the fronto-parallel plane and a plane that rotates aboutthe horizontal axis is the pitch axis. The angle a feature on thefronto-parallel makes as it rotates about an axis perpendicular to thefronto-parallel plane is the tilt axis.

Typical performances for camera-based barcode scanners are that they areable to detect barcode symbols within some range of distances as long asboth pitch and skew of the plane of the symbol are within the range ofplus or minus 45 degrees, while the tilt of the symbol can be arbitrary(between 0 and 360 degrees). The barcode scanner performance modelpredicts whether a given barcode symbol in a given position andorientation will be detected.

The barcode scanner performance model is coupled with a model of wherebarcodes would expect to be positioned and oriented. A barcode symbolpose model is the range of all positions and orientations, in otherwords poses, in which a barcode symbol will expect to be found. For thesecondary scanner, the barcode symbol pose model is itself a combinationof an article gripping model, which predicts how objects will be held bythe robotic pick and place system, as well as a barcode-item appearancemodel, which describes the possible placements of the barcode symbol onthe object. For the primary scanner, the barcode symbol pose model isitself a combination of the barcode-item appearance model, as well as aninbound-object pose model, which models the distribution of poses overwhich inbound articles are presented to the primary scanner. Thesemodels may be constructed empirically, modeled using an analyticalmodel, or approximate models can be employed using sphere models forobjects and a uniform distribution over the sphere as a barcode-itemappearance model.

In an embodiment for example, two objects, a bottle and a toothpastecontainer, represent the object set, and the barcode symbols are placedin fixed and known locations on all instances of these objects. With a3D model of these two objects, the known capabilities of the roboticpick and place system are used to generate a random sample of poses ofthe objects. These poses are with respect to the end-effector of therobotic pick and place system, typically a gripper.

FIG. 4 shows an example of a single simulated hold of a bottle 80, asheld by an end effector 82 of a vacuum-gripper-based robotic pick andplace system. FIG. 5 shows at 84 overlapping samples of the bottle 80shown in FIG. 4 . With these samples, and because the barcode symbolsare at fixed positions on the two articles, a set of poses of barcodesymbols are generated. FIG. 6 shows at 86 an example where thequadrilaterals represent 100 sampled barcode symbols. This samplerepresents the barcode symbol pose model for a secondary scanner. It isan approximation of a probability distribution over where barcodes wouldexpect to be found when the article is held by the robotic pick andplace system.

With these models, the barcode symbol pose model and the barcode scannerperformance model, optimization over all of the parameters of the systemis possible. FIG. 7 shows the resulting configuration, determined bysimulation of the above-described models, of a secondary scanner systemdetermined for the instance where the articles are a bottle andtoothpaste container. As shown in FIG. 7 , the system includes scanners90, 92, 94, 96 and 98, each of which is generally directed toward anobject area 88, while the position of each of the scanners 90-98 isselected to provide optimum planes, angles, tilts and views for theobjects in question. FIG. 7 shows a mechanical model of the actualizedsecondary scanner. In this instance the optimization criteria was theprobability of scan success. In instances where only one scanner can beemployed, the optimization criteria may be average scan time, in whichcase the optimization is over the sequence of poses in which to presentarticles to a secondary scanner so that the average scan time as a wholeis decreased.

In accordance with a further embodiment therefore, the invention may beused in connection with an object sortation system that yields a large(and very flexible) number of total collection bins, very low divertcosts per bin, throughput as high as that of a manual system, and a farsmaller need for manual labor to operate.

FIG. 8 , for example, shows a system 200 that includes an articulatedarm 202 with an end effector 204, an input area 206 in which objects arepresented for sortation, a primary perception system (e.g., a perceptionunit) 214 such as a camera for identifying objects to be sorted, and areceiving conveyor 208 for receiving objects to be sorted from any of ahuman worker, another conveyor, or an input pan. The system alsoincludes a non-sortable output chute 210 that leads to a non-sortableoutput bin 212 for providing objects that the system either could notidentify or could not sort for any other reason (e.g., could not graspor pick up).

In addition to the primary perception unit 214, the system also includesa drop perception system 216, which includes an open top and an openbottom, and a plurality of perception units (e.g., cameras or sensors asdiscussed above with reference to the previous embodiments) positionedwithin the perception system 216 that are aimed at the top, mid andlower central regions of the interior of the perception system 216. Theplurality of perception units, e.g., cameras, record perception data,e.g., images, of an object when it is dropped by the end effectorthrough the perception system 216. The drop perception system 216 mayalso include one or more sensors (e.g., laser sensors) at the top of thesystem 216 that detect when an object is dropped into the dropperception system 216. The plurality of perception units are designed tocollect a plurality of images of each object from multiple views to aidin identifying or confirming the identity of the dropped object.

The dropped object then falls into a first carriage 218 which isprovided on a track 220 on which the conveyor 218 may be movedautomatically between a first sortation stage 222 and a second sortationstage 224 on either side of the area in which the object was dropped.

The first sortation stage 222 includes a second carriage 226 that mayreceive objects from the first carriage 218, and which travels along atrack between two rows of collection bins 228 into which objects may bedumped along guide walls 230. The second sortation stage 224 includes athird carriage 232 that may receive objects from the first carriage 218,and which travels along a track between two rows of collection bins 234into which objects may be dumped along guide walls 236.

The system of FIG. 8 shows a system with two shuttle sort wings. When anobject is picked from the infeed conveyor, it is dropped onto the firstshuttle sorter 218. That shuttle sorter carries the object to one of twowings, drops the object in the carrier for that wing, and then movesback to home. Due to the limited travel, this back and forth operationmay be performed in the time it takes the articulated arm to pickanother object (assuming the articulated arm is picking objects atapproximately a human rate of throughput).

The drop perception system 216 includes a plurality of detection units(e.g., cameras or scanners as discussed above) that are directed towarda central path such that as an object falls through the unit 216,multiple views of the object will be captured by the multiple detectionunits. The drop perception system may also include lights as discussedabove with reference to the system of FIG. 2 .

FIGS. 9 and 10 show the detection units 300 directed toward the centralpath of the drop perception system 216 at varying angles. As also shown,a laser source bar 302 may direct laser illumination toward a sensor bar304 such that the drop perception system 216 may detect exactly when anobject enters the system 216. FIG. 10 shows at 306 diagrammaticillustrations of the multiple detector angles and fields of view foreach of the multiple detectors.

The plurality of additional perception systems may be positioned suchthat every surface of the object may be perceived regardless of theorientation of the object. In certain embodiments, the first perceptionsystem may provide perception data regarding a unique object identifier,and the plurality of additional perception systems may provideadditional perception data regarding any of confirming the objectidentifier or any additional object identifiers. In further embodiments,the first perception system may provide perception data regarding anobject shape, and the plurality of additional perception systems mayprovide additional perception data regarding a unique object identifierthat is confirmed as matching the object shape.

Those skilled in the art will appreciate that numerous modifications andvariations may be made to the above disclosed embodiments withoutdeparting from the spirit and scope of the present invention.

What is claimed is: 1-39. (Cancelled).
 40. A method of processingobjects using an end-effector of a programmable motion device, saidmethod comprising: receiving a plurality of objects at an input area;generating perception data regarding the plurality of objects at theinput area; generating a list of detected markings, each marking beingassociated with marking position coordinates; generating a list ofunidentified objects, each unidentified object being associated withobject position coordinates; registering the marking positioncoordinates with the object position coordinates to associate with eachmarking with an object of the plurality of objects; generating acandidate pick list that ranks objects for picking; and using theend-effector of the programmable motion device to grasp a selectedobject of the plurality of objects based on the candidate pick list. 41.The method of claim 40, wherein the plurality of objects are provided atthe input area in a bin.
 42. The method of claim 40, wherein theselected object is grasped by the end-effector on the marking associatedwith the selected object.
 43. The method of claim 40, wherein the methodfurther includes generating a map of the plurality of objects.
 44. Themethod of claim 40, wherein the method further includes generatingfurther perception data regarding the plurality of objects at the inputarea following removal of the selected object to confirm whether markingassociated with the selected object is still present at the input area.45. The method of claim 44, wherein the method further includesreturning the selected object to the input area if the markingassociated with the selected object is present in the further perceptiondata.
 46. The method of claim 40, wherein the method further includesrotating the selected object while grasping the selected object by theend-effector proximate a perception unit.
 47. The method of claim 40,wherein the method further includes transferring the selected object toa special outbound destination for special processing by humanpersonnel.
 48. The method of claim 40, wherein the method furtherincludes dropping the selected object into a drop perception system withan open top and an open bottom.
 49. The method of claim 40, wherein themethod further includes receiving the selected object in a carriage thatis adapted for movement toward a plurality of destination locations. 50.The method of claim 49, wherein the method further includes dropping theselected object from the carriage into a selected destination location.51. A method of processing objects using an end-effector of aprogrammable motion device, said method comprising: receiving aplurality of objects at an input area; generating perception dataregarding the plurality of objects at the input area; generating a listof unidentified objects, each unidentified object being associated withobject position coordinates; generating a map of the plurality ofobjects at the input area, said map including shaded segments associatedwith each of the plurality of objects; selecting a selected unidentifiedobject of the plurality of objects for grasping at the selected graspinglocation based on the map; and using the end-effector of theprogrammable motion device to grasp the unidentified selected object atthe selected grasping location.
 52. The method of claim 51, wherein theplurality of objects are provided at the input area in a bin.
 53. Themethod of claim 51, wherein the method further includes rotating theselected unidentified object while grasping the selected object by theend-effector proximate a perception unit.
 54. The method of claim 51,wherein the method further includes transferring the unidentifiedselected object to a special outbound destination for special processingby human personnel.
 55. The method of claim 51, wherein the methodfurther includes dropping the unidentified selected object into a dropperception system with an open top and an open bottom.
 56. The method ofclaim 51, wherein the method further incudes moving the unidentifiedselected object toward a plurality of perception systems that arrangedin a bowl-shaped arrangement.
 57. The method of claim 51, wherein themethod further includes receiving the selected object in a carriage thatis adapted for movement toward a plurality of destination locations. 58.The method of claim 57, wherein the method further includes dropping theselected object from the carriage into a selected destination location.59. An object processing system for processing objects using anend-effector of a programmable motion device, said object processingsystem comprising: an input area for receiving a plurality of objects; aprimary perception system for generating perception data regarding theplurality of objects at the input area; a central control system forgenerating a list of detected markings, each marking being associatedwith marking position coordinates, for generating a list of unidentifiedobjects, each unidentified object being associated with object positioncoordinates, and for registering the marking position coordinates withthe object position coordinates to associate with each marking with anobject of the plurality of objects; and a robotic pick and place systemfor generating a candidate pick list that ranks objects for picking andfor using the end-effector of the programmable motion device to grasp aselected object of the plurality of objects based on the candidate picklist.
 60. The object processing system as claimed in claim 59, whereinthe plurality of objects are provided at the input area in a bin. 61.The object processing system as claimed in claim 59, wherein the centralcontrol system further generates a map of the plurality of objects. 62.The object processing system as claimed in claim 59, wherein the objectprocessing system further includes a drop perception system with an opentop and an open bottom into which the selected object may be dropped.63. The object processing system as claimed in claim 59, wherein theobject processing system further includes a carriage that is adapted formovement toward a plurality of destination locations.
 64. The objectprocessing system as claimed in claim 63, wherein the plurality ofdestination locations are provided adjacent a track along which carriageis adapted to travel.